Section: New Results

Markov Random Fields

Hierarchical multitemporal and multiresolution classification in remote sensing imagery

Participants : Ihsen Hedhli, Josiane Zerubia [contact] .

This activity was conducted in collaboration with Dr. Gabriele Moser and Prof. Sebastiano B. Serpico (Department of Electrical, Electronic, and Telecommunications Engineering and Naval Architecture, DITEN, University of Genoa) [http://www.unige.it ] with partial financial support from CNES [http://www.cnes.fr ].

Markov random field (MRF), hierarchical classification, satellite image time series

The capability to monitor the Earth's surface, and especially urban and built-up areas, for environmental disasters such as floods or earthquakes, and to assess the ground impact and damage caused by such events, play important roles from multiple social, economic, and human viewpoints. Current and forthcoming satellite missions for Earth observation (EO; e.g., Pleiades, COSMO-SkyMed, TerraSAR-X, Sentinel) possess huge potential for such applications, as they allow a spatially distributed and temporally repetitive view of the monitored area at the desired spatial scales. In this framework, accurate and time-efficient classification methods using time series are especially important tools for supporting rapid and reliable assessment of the ground changes and damage induced by a disaster, in particular when an extensive area has been affected. Given the huge amount and variety of data available, the main difficulty is to find a classifier that takes into account multi-band, multi-resolution, multi-date, and possibly multi-sensor data.

This research addresses the problem of supervised classification at multiple spatial resolutions for multiple dates. The approach is based on the extension of recent methods proposed by DITEN and/or AYIN [4] , [5] , [6] . These methods focus on a supervised Bayesian classifier that combines joint class-conditional statistical modeling and a hierarchical Markov random field. The key idea of the proposed method is to combine the multiresolution modeling capabilities of this previous technique with a model for the temporal correlation among distinct images in a time series. For this purpose, a hierarchical spatio-temporal Markov random field model has been proposed that is aimed at fusing the pixel-wise, neighborhood, multiresolution, and temporal information associated with the input time series. Pixel-wise information is characterized through separate statistical modeling for each target class (e.g., vegetation, urban, etc.) by using a finite mixture model, estimated using a modified stochastic expectation maximization algorithm. Such a model is well suited to dealing with heterogeneous classes, and each mixture component may reflect the contribution of the different materials contained in a given class. At each considered resolution, the different input bands are statistically combined by using multivariate copulas, and the resulting statistical pixel-wise model is integrated in a hierarchical Markov random field based on a quad-tree structure. Among the different algorithms employed in the literature, we chose to use an exact estimator based on the marginal posterior mode (MPM). Specifically, a new formulation of MPM is developed to formalize, within the aforementioned hierarchical model, a `cascade' multi-date decision rule. Such a classifier is sufficiently flexible to take into account different types of data (e.g., multispectral, panchromatic, synthetic aperture radar). The method is being experimentally validated with data acquired over a given area at different resolutions (e.g., multiresolution Pleiades images), directly integrated at the different levels of the cascade hierarchical model. An example of a classification result is illustrated in Fig. 1 . Here, Pleiades multiresolution images (panchromatic resolution: 50 centimeters and multispectral resolution: 2 meters) acquired over Port-au-Prince quay (Haiti) on two different dates are considered. Spatially disjoint training areas were manually annotated. The classification has been performed with respect to 5 main classes: urban areas, natural landscape, sand, containers, and wet areas. A visual analysis of the resulting map suggests that the proposed approach achieves remarkable accuracy.

A multi-layer Markov model for change detection in temporally separated aerial image pairs

Participants : Praveer Singh, Josiane Zerubia [contact] .

This work was carried out in collaboration with Prof. Zoltan Kato from Institute of Informatics, University of Szeged, Hungary. [http://www.inf.u-szeged.hu/~kato/ ].

Multilayer Markov Random Fields (MRF), Histogram of Gradients (HOG), change detection, graph-cut optimization, aerial / satellite images.

In the proposed approach, we have tried to include both texture as well as pixel level information to build a three layer Markov model using the Histogram of Oriented Gradients (HOG) and the Gray Level Difference features on the topmost and bottommost layer respectively. Using a ground truth (GT) mask defined manually by an expert for each of the image pairs in the data set (obtained from the Hungarian Institute of Geodesy, Cartography and Remote Sensing), we employ a supervised technique to mark the initial set of pixels / sites as foreground or background. On the basis of the HOG difference and the Gray level difference feature vector corresponding to all the pixels in the image pair, a probability density function is fitted individually for the binary label set comprising of foreground and background labels using the GT. The probabilistic estimate is calculated using one training image pair for each data set. Using this probabilistic measure, a negative log likelihood is computed for each pixel (for both the features as well as the binary label set) which is then passed to the energy function of the proposed 3-layer MRF model. The final segmentation is obtained by minimizing the energy using a graph-cut algorithm, and subsequently a final foreground and background labelling is obtained over the combined layer. Figure 2 , shows aerial image pairs, one of them captured in 1984 by FOMI, Hungary (a) and the other one by GoogleEarth in 2007 (c). (b) is the ground truth and (d) is a combination of the hierarchical MRF based change detection (in red), ground truth (in green) and changes detected correctly (in yellow).

Graph-cut model for spectral-spatial classification of hyperspectral images

Participants : Aakanksha Rana, Yuliya Tarabalka [contact] .

Hyperspectral images, graph cut, multi-label alpha expansion, contextual information, energy minimization

The very high spatial and spectral resolution of the last generation of remote sensors provides rich information about every pixel in an image scene, hence opening new perspectives in classification, but also presenting the challenge of analysing high data volumes. While pixel-wise classification methods analyze each pixel independently, classification results can be significantly improved by including spatial information in a classifier.

In this work, we proposed a spectral-spatial method for hyperspectral image classification based on a graph cut. The classification task is expressed as an energy minimization problem on the spatio-temporal graph of image pixels, and is solved by using the graph-cut $\alpha$-expansion approach. The energy to optimize is computed as a sum of data and interaction energy terms, respectively. The data energy term is computed using the outputs of the probabilistic support vector machines classification. The second energy term, which expresses the interaction between spatially adjacent pixels in the eight-neighborhood, is computed by using dissimilarity measures between spectral vectors, such as vector norms, spectral angle map, or spectral information divergence. The performance of the proposed method was validated on hyperspectral images captured by the ROSIS and the AVIRIS sensors. Figure 3 compares classification results obtained by applying support vector machines and the proposed approach for the ROSIS hyperspectral image acquired over the University of Pavia. The new method yields higher classification accuracies when compared to the recent state-of-the-art approaches.